Self-Supervised Representation Learning for Evolutionary Neural Architecture Search

Recently proposed neural architecture search (NAS) algorithms adopt predictors to accelerate search. The capability of accurately predict the performance metrics is critical NAS, but obtaining training datasets for often time-consuming. How obtain a predictor with high prediction accuracy using small amount data central problem predictor-based NAS. Here, new encoding scheme first devised calculate graph edit distance architectures, which overcomes drawbacks existing vector-based schemes. To enhance predictive predictors, two self-supervised learning methods are pre-train embedding part generate meaningful representation architectures. method designs network-based model independent branches and utilizes different architectures as supervision force representations. Inspired by contrastive learning, second presents algorithm that feature vector proxy contrast positive pairs against negative pairs. Experimental results illustrate pre-trained can achieve comparable or superior compared their supervised counterparts only half samples. effectiveness further validated integrating into guided evolutionary (NPENAS) algorithm, achieves stateof-the-art on NASBench-101, NASBench-201, DARTS benchmarks